Detection and usability of personal electronic devices for field engineers

ABSTRACT

A system and method for improved detection and usability of personal electronic devices for field engineers is disclosed. A personal electronic device stores a list of commands in memory. The personal electronic device presents a user interface in a relative motion input mode, wherein the user interface in the relative motion mode is divided into a plurality of user interface zones. The personal electronic device receives relative motion input, wherein the relative motion input is received through one or more relative motion sensors of the personal electronic device. The personal electronic device determines whether the received relative motion input matches input associated with a respective command in a list of reference commands. In accordance with a determination that the received relative motion input matches input is associated with a respective command in the list of reference commands, the personal electronic device executes the respective command.

TECHNICAL FIELD

The disclosed embodiments relate generally to the field of electronicdevices, and in particular to user interfaces for electronic devices.

BACKGROUND

Computer devices have increasingly become smaller and more portable asthe technology improves. This improvement has led to widespread adoptionof personal computer devices (e.g., smartphones, tablet computers,wearable computer devices, and so on). These devices have becomeincreasingly important to users' lives.

Smaller, more portable electronic devices increasingly allow workers totake their personal electronic devices with them out of the office foruse in the field to collect data and improve productivity. Unfortunatelymany of these devices rely on a touchscreen as the primary inputmechanism, and in many cases field work is accomplished in locations andenvironments that make a touchscreen difficult to use. For example, coldenvironments might necessitate gloves or mittens, which rendertouchscreens less effective. As a result, it can be difficult to use apersonal electronic device in many real-world locations and contexts.

DESCRIPTION OF THE DRAWINGS

Some embodiments are illustrated by way of example and not limitation inthe figures of the accompanying drawings, in which:

FIG. 1 is a network diagram depicting a personal electronic device, inaccordance with an example embodiment, that includes various functionalcomponents.

FIG. 2 is a block diagram illustrating a personal electronic device, inaccordance with an example embodiment.

FIG. 3 illustrates a block diagram of an input analysis module, inaccordance with an example embodiment.

FIG. 4 illustrates an exemplary user interface of a personal electronicdevice in relative motion input mode, in accordance with an exampleembodiment.

FIG. 5 illustrates an exemplary embodiment of a personal electronicdevice in relative motion input mode, in accordance with an exampleembodiment.

FIG. 6 is a flow diagram illustrating a method, in accordance with anexample embodiment, for improving input detection and usability ofpersonal electronic devices when touch input is not the optimal inputmethod.

FIGS. 7A-7C are flow diagrams illustrating a method, in accordance withan example embodiment, for improving input detection and usability ofpersonal electronic devices when touch input is not the optimal inputmethod.

FIG. 8 is a block diagram illustrating an architecture of software, inaccordance with an example embodiment, which may be installed on any oneor more devices.

FIG. 9 is a block diagram illustrating components of a machine, inaccordance with an example embodiment.

Like reference numerals refer to corresponding parts throughout thedrawings.

DETAILED DESCRIPTION

The present disclosure describes methods, systems, and computer programproducts for improving input detection and usability of personalelectronic devices when touch input is not the optimal input method. Inthe following description, for purposes of explanation, numerousspecific details are set forth to provide a thorough understanding ofthe various aspects of different embodiments. It will be evident,however, to one skilled in the art, that the any particular embodimentmay be practiced without all of the specific details and/or withvariations permutations and combinations of the various features andelements described herein.

In some example embodiments, many personal electronic devices havetouchscreens that are used to receive input from a user. For example,smartphones, tablet computers, and smart watches all use a touchscreeninterface as the primary means for receiving input from a user. However,if the user has difficulty using a touchscreen to input data, it can bevery difficult for the user to use one of these personal electronicdevices effectively. For example, some users in the field wear gloves orhave greasy or dirty hands. In other examples, some users have adisability that prevents effective use of a touchscreen.

In some example embodiments, a personal electronic device can detectusability problems by the user. For example, if the personal electronicdevice detects a very large touch area on a touchscreen (e.g., the useris wearing large work gloves, mittens, and so on), the personalelectronic device determines that the user is having difficultyinputting data on the touchscreen. In another example, if the userfrequently has input errors (e.g., frequently has to undo commands), thepersonal electronic device also determines that touchscreen input isdifficult. In response to determining that touchscreen input isdifficult for a user, the personal electronic device switches into anon-touchscreen mode. In some example embodiments, the user can directlychoose to enter non-touchscreen mode.

In some example embodiments, non-touchscreen mode includes a specializeduser interface to allow easier use. Thus, in some example embodiments,in the non-touchscreen mode, the user interface is organized in to aseries of square sections. Navigation between the sections is easier andcontrollable through relative motion inputs. In some exampleembodiments, relative motion inputs include non-touch inputs such asvoice input or camera input. In other example embodiments, relativemotion inputs include moving or rotating the device relative to aprevious position. In some example embodiments, relative motion inputscan be input by any sensor on the personal electronic device with theexception of the touch screen. In some example embodiments, anotification is given to the user before switching into non-touch mode.In some example embodiments, the user directly selects switching intonon-touch mode.

In some example embodiments, once in non-touch mode, the personalelectronic device receives input from a user via one or more sensorsassociated with the personal electronic device. For example, thepersonal electronic device can detect movement or rotation of the devicethrough either an accelerometer or a gyroscope. In other exampleembodiments, the personal electronic device receives input from a userthrough a microphone (voice commands and so on) or a camera (usermotion).

In some example embodiments, the personal electronic device receives theinput from a user and then compares the received input to a plurality ofreference commands. Each respective reference command has one or moreinputs that are associated with the respective reference command. If amatch is found, the associated command is executed. If no match isfound, the personal electronic device then attempts to match thereceived input to its closest possible match in the list of referencecommands.

In some example embodiments, the personal electronic device determinesthe best match by identifying all the commands that are of the same typeof input (e.g., if the input command involved tilting the devicebackward, only commands with associated input that involve tilting ofthe device are compared). Then the personal electronic device determinesone command that best matches the input. For example, the devicecalculates the difference between the received input and the referencecommand input for each possible command and selects the command whoseinput has the smallest difference from the received input.

In some example embodiments, the personal electronic device executes thedetermined best match command. Once the best match command has beenexecuted, the personal electronic device determines whether the bestmatch command was the command the user intended. In some exampleembodiments, the personal electronic device determines whether the bestmatch command was the user-intended command by monitoring the subsequentuser activity. For example, if the user immediately reverses the action,the best match command is determined not to be the user-intendedcommand. However, if the user does not reverse the command and continuesto enter additional input, the personal electronic device determinesthat the best match command is the user-intended command.

In some example embodiments, in accordance with a determination that thebest match command is the user-intended command, the personal electronicdevice adds the new input to the input associated with the best matchcommand. For example, the personal electronic device detects a tilt offive degrees. The personal electronic device determines that no currentcommand is triggered by a tilt of five degrees. It then determines thatthe “activate currently selected icon” action is triggered by a tilt ofthree degrees and is the best match for the determined output. Thepersonal electronic device then executes the “activate currentlyselected icon” command. The user does not undo the command and thus thepersonal electronic device updates the input associated with the“activate currently selected icon” command to include any tilt within arange of three to five degrees.

FIG. 1 is a network diagram depicting a personal electronic device 120,in accordance with an example embodiment, that includes variousfunctional components. In some example embodiments, the personalelectronic device 120 is part of a client-server system 100 thatincludes the personal electronic device 120 and one or more third partyservers 150. One or more communications networks 110 interconnect thesecomponents. The communications network 110 may be any of a variety ofnetwork types, including local area networks (LANs), wide area networks(WANs), wireless networks, wired networks, the Internet, personal areanetworks (PANs), or a combination of such networks.

In some embodiments, as shown in FIG. 1, the personal electronic device120 is generally based on a three-tiered architecture, consisting of afront-end layer, an application logic layer, and a data layer. As isunderstood by skilled artisans in the relevant computer andInternet-related arts, each module or engine shown in FIG. 1 representsa set of executable software instructions and the corresponding hardware(e.g., memory and processor) for executing the instructions. To avoidunnecessary detail, various functional modules and engines that are notgermane to conveying an understanding of the various embodiments havebeen omitted from FIG. 1. However, a skilled artisan will readilyrecognize that various additional functional modules and engines may beused with a personal electronic device 120, such as that illustrated inFIG. 1, to facilitate additional functionality that is not specificallydescribed herein. Furthermore, the various functional modules andengines depicted in FIG. 1 may reside on a single server computer, ormay be distributed across several server computers in variousarrangements. Moreover, although the personal electronic device 120 isdepicted in FIG. 1 as having a three-tiered architecture, the variousembodiments are by no means limited to this architecture.

As shown in FIG. 1, the front-end layer consists of user interfacemodule(s) 122, which receives input from a user through one or moreinput systems (e.g., microphone, motion sensors, camera, and so on), andpresents the appropriate responses on one or more output systems (e.g.,display, speakers, and so on).

As shown in FIG. 1, the data layer includes one or more databases,including databases for storing data for users of the personalelectronic device 120, including user profile data 130 and command data132 (e.g., data listing the input associated with one or more commandsavailable through the personal electronic device 120).

In some embodiments, the user profile data 130 includes data associatedwith the user, including but not limited to user name, user age, userlocation, user activity data (e.g., applications and commands used bythe user), user schedule data (e.g., data about when the user is workingand when the user is likely to be unable to use a touchscreen), andother data related to and obtained from the user.

The command data 132 includes data that maps certain types of non-touchbased input to one or more commands provided by the personal electronicdevice 120. Each command has a list of inputs that will trigger it. Forexample, the command data 132 is a table and includes an entry for ascroll up command. The associated input is an accelerometer detectingsudden upward acceleration of the personal electronic device 120 in therange of 0.5-1.0 inches per second. Another example command in thecommand data 132 is page up, which is associated with a detected upwardacceleration of in the range of two to three inches per second. Thus,the personal electronic device 120 will match received input against thestored command data 132 and execute one of the two different actions.

The personal electronic device 120 provides a broad range of otherapplications and services that allow users the opportunity to share andreceive information, often customized to the interests of the users.

In some embodiments, the application logic layer includes variousapplication server modules, which, in conjunction with the userinterface module(s) 122, generate various user interfaces to receiveinput from and deliver output to a user. In some embodiments, individualapplication modules are used to implement the functionality associatedwith various applications, services, and features of the personalelectronic device 120. For instance, a messaging application, such as anemail application, an instant messaging application, or some hybrid orvariation of the two, may be implemented with one or more applicationmodules. Similarly, a web browser enabling members to view web pages maybe embodied with one or more application modules. Of course, otherapplications or services that utilize a detection module 124 and aninput analysis module 126 may be separately embodied in their ownapplication modules.

In addition to the various application server modules, the applicationlogic layer includes a detection module 124 and an input analysis module126. As illustrated in FIG. 1, in some embodiments, the detection module124 and the input analysis module 126 are embodied as modules thatoperate in conjunction with various application modules. For instance,any number of individual application modules can invoke thefunctionality of the detection module 124 and the input analysis module126 to receive user input and analyze it. However, in variousalternative embodiments, the detection module 124 and the input analysismodule 126 may be embodied as their own application modules such thatthey operate as a stand-alone application.

Generally, the detection module 124 detects that a user is not able toefficiently use the touchscreen to input user commands. In some exampleembodiments, this is based on the size of the user's contact with atouchscreen (e.g., very large contact spaces may indicate that the useris wearing gloves or mittens), trouble detecting user input (e.g., basedon dirt or oil on the user's hands), user disability (e.g., unable tohold hands still), or direct user instruction (e.g., the user selects anoption indicating that using the touchscreen is currently difficult). Inaccordance with a determination that the user is not able to efficientlyuse the touchscreen, the personal electronic device 120 changes to anon-touch mode. In some example embodiments, the personal electronicdevice 120 was originally in a touch mode or normal operation mode.

In some example embodiments, the detection module 124 switches thepersonal electronic device 120 into non-touch mode by changing thedisplayed user interface into a user interface that is organized into aseries of square sections, for easy navigation.

The input analysis module 126 stores a list of possible commands thatthe personal electronic device 120 is able to execute. Each possiblecommand also includes a list of one or more user inputs that areassociated with the possible command and will result in commandexecution.

In some example embodiments, the input analysis module 126 receivesinput data from the detection module 124. In some example embodiments,the input data includes more than one component. The input analysismodule 126 compares the input data against the one or more storedcommands.

In some example embodiments, the input analysis module 126 finds anexact match (e.g., a command that lists input that is a direct match forthe received input). For example, an input is received that is amovement of the personal electronic device 120 accelerating upward inthree inches per second. The input analysis module 126 determines if anyof the listed commands include raising the personal electronic device120 in three inches per second as the input associated with the command.

In some example embodiments, in accordance with a determination that thereceived input matches the input for a command stored in the list ofcommands, the input analysis module 126 causes the matching command tobe executed. In accordance with a determination that the received inputdoes not match any stored command, the input analysis module 126determines the closest matching command. In some example embodiments,the closest match is determined based on a fuzzy matching algorithm. Forexample, the input analysis module 126 can determine one or morecommands with inputs that are of the same type as the received userinput. Thus, if the received user input involves rotating the personalelectronic device 120, the input analysis module 126 will identify oneor more command with associated inputs that involve rotation of thepersonal electronic device 120.

In some example embodiments, the input analysis module 126 can determinethe closeness of the stored associated input to the received user input.For example, if the received user input is a rotation of the personalelectronic device 120 back fifteen degrees, then a command with anassociated input range of rotation back five to ten degrees would be acloser match than a command with an associated input range of rotationforward five to ten degrees.

In some example embodiments, once the best matching command isidentified, the input analysis module 126 causes the personal electronicdevice 120 to execute the best matching command. In some exampleembodiments, the input analysis module 126 monitors user response to theexecuted best matching command to determine if the user accepts thecommand (e.g., continues to use the device without reversing thecommand) or immediately reverses it (e.g., an immediate reversal wouldindicate that the user did not wish the best matching command to beexecuted).

In some example embodiments, in accordance with a determination that theuser accepts the best matching command, the input data associated withthe best matching command is updated to extend its acceptable range toinclude the received user input.

In some example embodiments, the user can manually update the list ofcommands stored at the personal electronic device 120 or an inputassociated with an already stored command. In some example embodiments,the user enters a command editing mode (e.g., by selecting an optionavailable through the user interface), selects a specific command, andthen enters new or updated associated input data for the selectedcommand (either through selecting input on a list of inputs or byactually causing the associated input). The list of commands is thenupdated on the personal electronic device 120 and stored.

In some example embodiments, a third party server 150 stores user data152. This user data 152 can incorporate any information about the user,including, but not limited to, user preferences, user history, userlocation, user demographic information, and command input data for theuser. In some example embodiments, the user can switch from one personalelectronic device 120 to a different computer device and import all therelevant user profile data from the user data 152 stored at the thirdparty server 150. In this way, the user's command data will be availableat the new device automatically.

FIG. 2 is a block diagram illustrating the personal electronic device120, in accordance with an example embodiment. The personal electronicdevice 120 typically includes one or more processing units (CPUs) 202,one or more network interfaces 210, a memory 212, and one or morecommunication buses 214 for interconnecting these components. Thepersonal electronic device 120 includes a user interface 204. The userinterface 204 includes a display 206 and optionally includes an input208, such as a keyboard, mouse, touch-sensitive display, movementsensors (e.g., accelerometer or gyroscope), camera, or other inputmeans. Furthermore, some personal electronic devices 120 use amicrophone and voice recognition to supplement or replace the keyboard.

Memory 212 includes high-speed random-access memory, such as dynamicrandom-access memory (DRAM), static random-access memory (SRAM), doubledata rate random-access memory (DDR RAM), or other random-accesssolid-state memory devices, and may include non-volatile memory, such asone or more magnetic disk storage devices, optical disk storage devices,flash memory devices, or other non-volatile solid-state storage devices.Memory 212 may optionally include one or more storage devices remotelylocated from the CPU(s) 202. Memory 212, or alternately, thenon-volatile memory device(s) within memory 212, comprise(s) anon-transitory computer readable storage medium.

In some embodiments, memory 212 or the computer readable storage mediumof memory 212 stores the following programs, modules, and datastructures, or a subset thereof:

-   -   an operating system 216 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a network communication module 218 that is used for connecting        the personal electronic device 120 to other computers via the        one or more network interfaces 210 (wired or wireless) and one        or more communication networks, such as the Internet, other wide        area networks, local area networks, metropolitan area networks,        etc.;    -   a display module 220 for enabling the information generated by        the operating system 216 and application module(s) 222 to be        presented visually on the display 206;    -   one or more application module 222 for handling various aspects        of providing the services associated with the personal        electronic device 120, including but not limited to:        -   an input analysis module 126 for receiving input components            and determining whether the input components match any of            the input components associated with commands in the list of            commands stored in the command data 132;        -   a detection module 124 for detecting, based on received            input, that a user is not able to efficiently use the            touchscreen to input user commands, and wherein this            detection is based on detecting one or more input anomalies            in the received user input, including but not limited to:            size of input contact, steadiness of input contact, strength            of input contact (e.g., whether input is being disrupted by            dirt and/or oil on the user's hands);        -   a storage module 226 for storing and managing a list of            commands and the associated input data in the command data            132;        -   a presentation module 224 for causing a user interface to be            presented on a screen associated with the personal            electronic device 120 and updating the user interface based            on the operating mode of the personal electronic device 120;        -   a mode change module 228 for changing the operating mode of            the personal electronic device 120 from touch input mode to            relative motion input mode or from relative motion input            mode to touch input mode;        -   a reception module 230 for receiving user input from a            sensor associated with the personal electronic device 120;        -   an execution module 232 for causing a selected command to be            executed on the personal electronic device 120;        -   an update module 234 for updating command data stored for            one or more commands;        -   an anomaly detection module 236 for detecting anomalies in            user input (e.g., the size of the contact, steadiness of the            contact, and so on); and    -   data module(s) 240, for storing data relevant to the personal        electronic device 120, including but not limited to:        -   user profile data 130 for storing profile data related to a            user of the personal electronic device 120;        -   command data 132, including data related to commands that            can be executed by the personal electronic device 120 and            one or more associated inputs (or range of inputs) that will            result in the command being executed; and        -   command use history data 246 including a list of all used            commands and the number of instances that each command is            used.

FIG. 3 illustrates a block diagram of the input analysis module 126, inaccordance with an example embodiment. According to one embodiment, oneor more sensors 302 of the personal electronic device 120 detect inputfrom a user. The sensors 302 transfer the detected input to the inputlibrary engine 304. The input library engine 304 receives the input datafrom the one or more sensors 302 and converts the data into input datathat can be used by the input analysis module 126. For example, the datareceived from the sensors 302 is raw data from the sensors 302 and isconverted into a form useful to the rest of the input analysis module126.

In some example embodiments, once the data has been converted intouseful input data from by the input library engine 304, the input datais then transferred to the input matching module 306. The input matchingmodule 306 uses the received input data to build an input model 308 ofthe input. Once the input model 308 has been built, the input matchingmodule 306 uses the stored command data 132 to find a command withassociated inputs that match the received input data.

In some example embodiments, the input matching module 306 finds anexact match in the input data stored in the command data 132. Forexample, the received input is a ten degree tilt to the left; the inputanalysis module 126 determines that a first command stored in thecommand data 132 defines associated input as any left tilt within therange of five degrees to twelve degrees. In this case the input analysismodule 126 determines that the received input matches the range of inputassociated with the first command and transmits the first command to theevent emitter 310.

In some example embodiments, the event emitter 310 transmits theselected command to the user interface (or system that controls the userinterface) where the command is executed.

In some example embodiments, the input matching module 306 does not findan exact match. Continuing the above example, if the received input wasa left tilt of thirty degrees, it would no longer match the input range(five degrees to twelve degrees) associated with the first command. Theinput matching module 306 would then need to use the input model 308that was constructed to find a closest matching command. In some exampleembodiments, the input matching module 306 determines the best matchingcommand by using fuzzy matching algorithms.

In some example embodiments, the input matching module 306 woulddetermine a command match score for each potential matching command. Thecommand with the highest potential matching score would then be selectedas the best matching command and selected for execution. In some exampleembodiments, command match scores are selected based on whether theinputs are of the same type. For example, a command with an associatedinput based on leftward tilting of the personal electronic device (e.g.,the personal electronic device 120 of FIG. 1) would be a better match,and thus have a higher matching score, with received input dataindicating a left tilt than with a command with associated input thatincludes microphone detection of sound.

In some example embodiments, the command match score is based on thedirection of input. For example, if the received input data is movementof the personal electronic device 120 up three inches, then a commandwith a reference input upward will have a higher score than a commandwith reference input downward. Similarly, the command match score isbased on the degree. Thus, a command with reference input data that is afive-degree tilt leftward will have a high match score based on inputthat is a ten-degree tilt leftward than a command with reference inputdata that is thirty degrees leftward.

In some example embodiments, the learning module 314 receives input dataform the input library engine 304 that represents new input to beassociated with a new command in the command data 132 or updated inputdata for an existing command.

FIG. 4 illustrates an exemplary user interface of a personal electronicdevice (e.g., personal electronic device 120 of FIG. 1) in relativemotion input mode, in accordance with an example embodiment. As can beseen, the user interface 400 shown in FIG. 4 has been formatted to bevery simple. All the user interface elements have been standardized intozones that are simple rectangles and all the zones have been arranged inbasic rows and columns. Thus, a limited number of commands can be usedto easily and quickly navigate through the user interface 400.

In some example embodiments, the interface 400 includes a home button402, two option buttons (404 and 406), four title buttons (408, 410,412, and 414), three menu buttons (416, 418, and 420), and six contentboxes (422, 424, 426, 428, 430, and 432). These components are arrangedinto five columns and at least five rows. In some example embodiments,information about the components in the user interface 400 can be storedin a simple data structure. For example:

var matrix = [ [‘home’, null, null, ‘option 1’, ‘option 2’], [null,‘title 1’, ‘title 2’, ‘title 3’, ‘title 4’], [‘menu 1’, ‘box 1’, ‘box1’, ‘box 2’, ‘box 2’], [‘menu 2’, ‘box 1’, ‘box 1’, ‘box 2’, ‘box 2’],[‘menu 3’, ‘box 1’, ‘box 1’, ‘box 2’, ‘box 2’], [‘null’, ‘box 3’, ‘box3’, ‘box 4’, ‘box 4’], [‘null’, ‘box 5’, ‘box 5’, ‘box 6’, ‘box 6’] ]

Using this simple data structure, the user interface 400 is relativelysimple to navigate with a limited number of possible user inputs. Insome example embodiments, other data structures or storage structuresare used to easily navigate the user interface 400.

FIG. 5 illustrates an exemplary embodiment of a personal electronicdevice 500 in relative motion input mode, in accordance with an exampleembodiment. As can be seen, the particular personal electronic device500 includes a display 502 (e.g., a touchscreen display). However,because the personal electronic device 500 is in relative motion inputmode, the displayed user interface includes a series of components(e.g., icons, buttons, data, and so on) as rectangles in a pattern ofrows and columns. Currently component 506 has been selected and the userinterface reflects the selected component 506 by giving the selectedcomponent 506 a distinct visual appearance (represented in the figure bydifferent shading).

In some example embodiments, the personal electronic device 500 inrelative motion input mode allows input from one or more differentmotions. For example, the user can roll 510 the personal electronicdevice 500 along the y-axis of the personal electronic device 500.Another example input is to pitch 512 the personal electronic device 500around the x-axis. The personal electronic device 500 can also detecttapping (e.g., or other sudden movement or acceleration) 514 of thepersonal electronic device 500 or shaking 516 of the device 500. Each ofthese potential inputs can be mapped by the personal electronic device500 to specific commands. When one of these inputs is detected, thepersonal electronic device 500 can then execute the associated command.

FIG. 6 is a flow diagram illustrating a method 600, in accordance withan example embodiment, for improving input detection and usability ofpersonal electronic devices when touch input is not the optimal inputmethod. Each of the operations shown in FIG. 6 may correspond toinstructions stored in a computer memory or computer-readable storagemedium. In some embodiments, the method 600 described in FIG. 6 isperformed by the personal electronic device (e.g., personal electronicdevice 120 in FIG. 1).

In some embodiments, the method 600 is performed at a personalelectronic device (e.g., personal electronic device 120 in FIG. 1)including one or more processors and memory storing one or more programsfor execution by the one or more processors.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) is initially in a touch inputmode. In some example embodiments, touch input mode is the standard modefor a personal electronic device that includes a touchscreen and meansthat the primary way of interacting with the device would be throughtouch input received through the touchscreen (other input means maystill be available).

In some example embodiments, while in touch input mode, the personalelectronic device (e.g., personal electronic device 120 in FIG. 1)displays (602) a user interface to a user. While the device is in touchinput mode, each application displays its standard or preferred userinterface. While displaying the user interface, the personal electronicdevice receives (604) user input. In some example embodiments, userinput includes touch input on a touch screen (e.g., finger contacts thattap, swipe, and so on.)

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) detects (606) anomalies inreceived input (touch input). Examples of anomalies include larger thannormal contact size (e.g., the user is wearing gloves or mittens),unsteady contact (e.g., the user has difficulty holding hands or fingerssteady), frequent input error (e.g., the user routinely has to undocommands and give the same or similar input again seeking a differentcommand), weakly detected input (fingers or screen that is dirty orotherwise compromised), and so on.

In some example embodiments, in response to detecting input anomalies,the personal electronic device (e.g., personal electronic device 120 inFIG. 1) changes (608) the operating mode from touch input mode (e.g.,the mode where input primarily comes from a touch screen) to therelative motion input mode (e.g., the operating mode where inputprimarily comes from non-touch based systems.) In some exampleembodiments, the personal electronic device (e.g., personal electronicdevice 120 in FIG. 1) causes a notification to be displayed indicatingthe changes. In other example embodiments, the personal electronicdevice queries the user (e.g., through an onscreen prompt) whether theuser accepts the operating mode switch. If the user rejects the switch,no change is made.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) then updates (608) the userinterface to a user interface type configured to be easily used in therelative motion input mode. For example, the elements of the userinterface are resized and rearranged into a grid arrangement and changedin shape to be rectangular. In this way, the user interface can benavigated more easily with relatively few commands.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) then receives (610) relativemotion input via one or more sensors (e.g., gyroscope, accelerometer,microphone, camera) associated with the personal electronic device. Thepersonal electronic device analyzes the received relative motion inputto determine (612) whether the received input matches the inputassociated with a respective command in a stored list of commands. Forexample, the personal electronic device (e.g., personal electronicdevice 120 in FIG. 1) stores a list of all commands it currently canreact to and also stores the specific input that will cause each commandto execute.

In accordance with a determination that the received relative motioninput matches input associated with a respective stored command, thepersonal electronic device (e.g., personal electronic device 120 inFIG. 1) executes (614) the respective stored command.

FIG. 7A is a flow diagram illustrating a method, in accordance with anexample embodiment, for improving input detection and usability ofpersonal electronic devices when touch input is not the optimal inputmethod. Each of the operations shown in FIG. 7A may correspond toinstructions stored in a computer memory or computer-readable storagemedium. Optional operations are indicated by dashed lines (e.g., boxeswith dashed-line borders). In some embodiments, the method described inFIG. 7A is performed by the personal electronic device (e.g., personalelectronic device 120 in FIG. 1). However, the method described can alsobe performed by any other suitable configuration of electronic hardware.

In some embodiments, the method is performed at a personal electronicdevice (e.g., personal electronic device 120 in FIG. 1) including one ormore processors and memory storing one or more programs for execution bythe one or more processors.

In some example embodiments, while the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) is operating (702) in a touchinput mode, the personal electronic device receives (704) user inputthrough a touchscreen associated with the personal electronic device.For example, the user taps or swipes on a touchscreen of the device.

In some example embodiments, while analyzing the received input, thepersonal electronic device (e.g., personal electronic device 120 inFIG. 1) detects (706) one or more touch input anomalies in the receiveduser input. As noted above, examples of anomalies include larger thannormal contact size (e.g., the user is wearing gloves or mittens),unsteady contact (e.g., the user has difficulty holding hands or fingerssteady), frequent input error (e.g., the user routinely has to undocommands and give the same or similar input again seeking a differentcommand), weakly detected input (fingers or screen that is dirty orotherwise compromised), and so on. Thus, an anomaly is detected when theuser is having difficulty entering input into the personal electronicdevice (e.g., personal electronic device 120 in FIG. 1) via atouchscreen interface.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1), in response to detecting(708) one or more touch anomalies in the received user input, displays(710) (or causes to be displayed) an operating mode change notificationon a display associated with the personal electronic device. Forexample, the screen of a tablet computer displays a pop-up windownotifying a user that the operating mode of the tablet is going tochange from input touch mode to non-input touch mode. In other exampleembodiments, the user interface displays a message in the message statusbar because a pop-up window can interrupt a user's work flow, which isgenerally undesirable.

In other example embodiments, in response to detecting (708) one or moretouch anomalies in the received user input, the personal electronicdevice (e.g., personal electronic device 120 in FIG. 1) displays (712) aprompt message on a display associated with the personal electronicdevice, wherein the prompt message suggests a change of operating mode.For example, the personal electronic device (e.g., personal electronicdevice 120 in FIG. 1) causes a popup to be displayed with the question“Would you like to switch into non-touch mode?” and includes yes and nobuttons as well as directions for accepting or rejecting using relativemotion input (e.g., tilt right for yes and tilt left for no). Thus, theuser is able to determine whether to enter the relative motion inputmode.

In other example embodiments, a status bar can inform the user how toenter the relative input mode without displaying a prompt or other popup. For example, the personal electronic device (e.g., personalelectronic device 120 in FIG. 1) detects that the relative input mode ispotentially needed and causes the status bar to display “Tilt left onceand back twice to enter a non-touch mode.” The user can then follow thedirections if the user wishes to enter the relative input mode.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) receives (714) inputindicating user acceptance of the change of operating mode. For example,the user selects the “yes” button by tapping it or by tilting the deviceto the right. In other example embodiments, directions to enter therelative input mode are displayed on the status bar and the personalelectronic device (e.g., personal electronic device 120 in FIG. 1)detects input matching the directions displayed on the status bar and inresponse determines that the user accepts changing the operating mode.

In some example embodiments, in response to receiving input indicatinguser acceptance of the change of operating mode, the personal electronicdevice (e.g., personal electronic device 120 in FIG. 1) changes (716)the operating mode of the personal electronic device from a touch inputmode to a relative motion input mode. For example, the user inputindicating user acceptance includes the user following a set ofdirections displayed to allow the user to indicate acceptance. In otherexample embodiments, the personal electronic device (e.g., personalelectronic device 120 in FIG. 1) changes the operating modeautomatically, without waiting for user approval, as soon as inputirregularities or anomalies occur.

In some example embodiments, in response changing the operating mode ofthe personal electronic device from a touch input mode to a relativemotion input mode, the personal electronic device (e.g., personalelectronic device 120 in FIG. 1) updates (718) the displayed userinterface to a relative motion input based user interface. The relativemotion input based user interface is arranged in a direct simple format(e.g., interface components are made into rectangular boxes arranged incolumns and rows) to facilitate ease of navigation throughout the userinterface.

FIG. 7B is a flow diagram illustrating a method for improving inputdetection and usability of personal electronic devices when touch inputis not the optimal input method, in accordance with someembodimentations. Each of the operations shown in FIG. 7B may correspondto instructions stored in a computer memory or computer-readable storagemedium. Optional operations are indicated by dashed lines (e.g., boxeswith dashed-line borders). In some embodimentations, the methoddescribed in FIG. 7B is performed by the personal electronic device(e.g., personal electronic device 120 in FIG. 1). However, the methoddescribed can also be performed by any other suitable configuration ofelectronic hardware.

In some embodimentations the method is performed at a personalelectronic device (e.g., personal electronic device 120 in FIG. 1)including one or more processors and memory storing one or more programsfor execution by the one or more processors.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) stores (720) a list ofcommands in memory associated with the personal electronic device,wherein each stored command includes a function executable by thepersonal electronic device and associated input that causes the commandto be executed. For example, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) includes a table in memory.Each entry in the table represents a command that is available to a user(e.g., move up in the user interface, move down, select an item, goback, undo, copy, paste, and so on).

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) presents (722), on a displayassociated with the personal electronic device, a user interface in arelative motion input mode, wherein the user interface in a non-touchmode is divided into a plurality of user interface zones. In someexample embodiments, the display associated with the personal electronicdevice is a touchscreen. For example, the user interface displays aseries of interface components in specific user interface zones that arerepresented by regular rectangular shapes arranged in rows and columnsfor easy navigation.

In some example embodiments, the user interface used when the personalelectronic device (e.g., personal electronic device 120 in FIG. 1) inrelative motion input operating mode is predetermined (e.g., the creatorof an app designates a user interface for non-touch operating mode). Inother example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) automatically generates a userinterface for use in relative motion input operating mode.

For example, the personal electronic device (e.g., personal electronicdevice 120 in FIG. 1) analyzes the standard (touch input mode) userinterface to identify one or more user interface elements (e.g.,buttons, inputs, icons, images, and so on). In some example embodiments,the personal electronic device (e.g., personal electronic device 120 inFIG. 1) assigns each identified element a regularly sized rectangle andplaces it in a grid of rows and columns. The user interface is displayedwith all the interface elements in the grid.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) receives (724) relative motioninput, wherein relative motion input is received through one or morenon-touch sensors of the personal electronic device. In some exampleembodiments, the non-touch sensors of the personal electronic deviceinclude one or more of a gyroscope, an accelerometer, a microphone, anda camera. In some example embodiments, the received input includes oneof detected shaking of the personal electronic device (e.g., personalelectronic device 120 in FIG. 1), raising or lowering the personalelectronic device (e.g., personal electronic device 120 in FIG. 1),tilting the device on any of the three dimensional axis, voice commands,visual cues, and so on.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) determines (726) whether thereceived relative motion input matches input associated with arespective command in the list of reference commands. To accomplishthis, the personal electronic device (e.g., personal electronic device120 in FIG. 1) converts sensor data into discrete input data. Input datadescribes the type of input (e.g., motion, sound, and so on) and thedegree (e.g., five degrees, shaking for five seconds, certain volumelevels or sounds, and so on).

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) then compares the discreteinput data to the input data stored in the list of stored commands. Ifthe discrete input data matches input data stored in the list of storedcommands, the personal electronic device (e.g., personal electronicdevice 120 in FIG. 1) determines that the received relative motion inputmatches input associated with a respective command in the list ofreference commands.

In some example embodiments, in accordance with a determination that thereceived relative motion input matches input associated with arespective command in the list of reference commands, the personalelectronic device (e.g., personal electronic device 120 in FIG. 1)executes (728) the respective command. For example, if the relativemotion input matches input associated with the “paste” command, thepersonal electronic device (e.g., personal electronic device 120 inFIG. 1) then pastes the appropriate content.

In some example embodiments, before receiving relative motion input, afirst user interface zone of the plurality of user interface zones isvisually distinguished as selected, and wherein executing the respectivecommand comprises, the personal electronic device (e.g., personalelectronic device 120 in FIG. 1) changing the visually distinguishedinterface zone from the first user interface zone to a second userinterface zone based on the respective command.

In some example embodiments, relative motion input represents a changein the personal electronic devices position or rotation. In otherexample embodiments, relative motion input represents motion of a userrelative to the personal electronic device.

FIG. 7C is a flow diagram illustrating a method for for improving inputdetection and usability of personal electronic devices when touch inputis not the optimal input method, in accordance with someembodimentations. Each of the operations shown in FIG. 7C may correspondto instructions stored in a computer memory or computer-readable storagemedium. Optional operations are indicated by dashed lines (e.g., boxeswith dashed-line borders). In some embodimentations, the methoddescribed in FIG. 7C is performed by the personal electronic device(e.g., personal electronic device 120 in FIG. 1). However, the methoddescribed can also be performed by any other suitable configuration ofelectronic hardware.

In some embodimentations the method is performed at a personalelectronic device (e.g., personal electronic device 120 in FIG. 1)including one or more processors and memory storing one or more programsfor execution by the one or more processors.

In some example embodiments, in accordance with a determination that thereceived non-touch user input does not match (730) the stored associatedinput with a respective command in the list of reference commands, thepersonal electronic device (e.g., personal electronic device 120 inFIG. 1) determines (732) a closest matching command. A closest matchingcommand is determined by generating a match score for one or morecommands. In some example embodiments, a match score for a respectivecommand is generated by comparing the received input with the inputassociated with the respective command to determine the type of input,the direction of the input, the timing of the input, and the degree ofthe input. The command with the highest matching score is then selectedas the closest matching command.

In some example embodiments, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) then executes (734) theclosest matching command.

In some example embodiments, after executing the personal electronicdevice (e.g., personal electronic device 120 in FIG. 1) monitors (736)future user inputs to determine whether an undo command was received.For example, once a closest matching command is executed, the personalelectronic device (e.g., personal electronic device 120 in FIG. 1) needsto determine whether the matching command was, in fact, the intendedcommand. If an undo command is detected immediately, the personalelectronic device (e.g., personal electronic device 120 in FIG. 1)determines that the closest matching command was not the intendedcommand. Once a predetermined amount of time passes (e.g., thirtyseconds) or other commands are received, the personal electronic device(e.g., personal electronic device 120 in FIG. 1) determines that theclosest matching command was the intended command.

In some example embodiments, in accordance with a determination that noundo command was received, the personal electronic device (e.g.,personal electronic device 120 in FIG. 1) updates (738) the input dataassociated with the closest matching command to include the receiveduser input. For example, if the received input was a five degree tilt tothe left of the personal electronic device (e.g., personal electronicdevice 120 in FIG. 1) and the closest matching command has associatedinput of a ten degree tilt to the left of the personal electronic device(e.g., personal electronic device 120 in FIG. 1), the input for theclosest matching command is updated to include the range five degrees toten degrees tilt to the left. Thus, the input for a command is updatedto adapt to a user's particular preferences without the user's explicitcommand.

In some example embodiments, input data associated with a stored commandincludes a range of possible inputs. For example, an input for the “up”command is a movement of the device upwards a distance in the range ofone inch to three inches. Thus, any input in that range will trigger the“up” command.

In some example embodiments, updating the input data associated with theclosest match command to include the received user input data comprisesextending the range of possible inputs to cover the received user input.

Software Architecture

FIG. 8 is a block diagram illustrating an architecture of software 800,in accordance with an example embodiment, which may be installed on anyone or more of the devices of FIG. 1 (e.g., the personal electronicdevice 120). FIG. 8 is merely a non-limiting example of a softwarearchitecture and it will be appreciated that many other architecturesmay be embodied to facilitate the functionality described herein. Thesoftware 800 may be executing on hardware such as a machine 900 of FIG.9 that includes processors 910, memory 930, and I/O components 950. Inthe example architecture of FIG. 8, the software 800 may beconceptualized as a stack of layers where each layer may provideparticular functionality. For example, the software 800 may includelayers such as an operating system 802, libraries 804, frameworks 806,and applications 808. Operationally, the applications 808 may invokeapplication programming interface (API) calls 810 through the softwarestack and receive messages 812 in response to the API calls 810.

The operating system 802 may manage hardware resources and providecommon services. The operating system 802 may include, for example, akernel 820, services 822, and drivers 824. The kernel 820 may act as anabstraction layer between the hardware and the other software layers.For example, the kernel 820 may be responsible for memory management,processor management (e.g., scheduling), component management,networking, security settings, and so on. The services 822 may provideother common services for the other software layers. The drivers 824 maybe responsible for controlling and/or interfacing with the underlyinghardware. For instance, the drivers 824 may include display drivers,camera drivers, Bluetooth® drivers, flash memory drivers, serialcommunication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi®drivers, audio drivers, power management drivers, and so forth.

The libraries 804 may provide a low-level common infrastructure that maybe utilized by the applications 808. The libraries 804 may includesystem libraries (e.g., C standard library) 830 that may providefunctions such as memory allocation functions, string manipulationfunctions, mathematic functions, and the like. In addition, thelibraries 804 may include API libraries 832 such as media libraries(e.g., libraries to support presentation and manipulation of variousmedia formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, or PNG),graphics libraries (e.g., an OpenGL framework that may be used to render2D and 3D in a graphic content on a display), database libraries (e.g.,SQLite that may provide various relational database functions), weblibraries (e.g., WebKit that may provide web browsing functionality),and the like. The libraries 804 may also include a wide variety of otherlibraries 834 to provide many other APIs to the applications 808.

The frameworks 806 may provide a high-level common infrastructure thatmay be utilized by the applications 808. For example, the frameworks 806may provide various graphic user interface (GUI) functions, high-levelresource management, high-level location services, and so forth. Theframeworks 806 may provide a broad spectrum of other APIs that may beutilized by the applications 808, some of which may be specific to aparticular operating system or platform.

The applications 808 include a home application 850, a contactsapplication 852, a browser application 854, a book reader application856, a location application 858, a media application 860, a messagingapplication 862, a game application 864, and a broad assortment of otherapplications, such as a third party application 866. In a specificexample, the third party application 866 (e.g., an application developedusing the Android™ or iOS™ software development kit (SDK) by an entityother than the vendor of the particular platform) may be mobile softwarerunning on a mobile operating system such as iOS™, Android™, Windows®Phone, or other mobile operating systems. In this example, the thirdparty application 866 may invoke the API calls 810 provided by theoperating system 802 to facilitate functionality described herein.

Example Machine Architecture and Machine-Readable Medium

FIG. 9 is a block diagram illustrating components of a machine 900,according to some example embodiments, able to read instructions from amachine-readable medium (e.g., a machine-readable storage medium) andperform any one or more of the methodologies discussed herein.Specifically, FIG. 9 shows a diagrammatic representation of the machine900 in the example form of a computer system, within which instructions925 (e.g., software, a program, an application, an applet, an app, orother executable code) for causing the machine 900 to perform any one ormore of the methodologies discussed herein may be executed. Inalternative embodiments, the machine 900 operates as a stand-alonedevice or may be coupled (e.g., networked) to other machines. In anetworked deployment, the machine 900 may operate in the capacity of aserver machine or a client machine in a server-client networkenvironment, or as a peer machine in a peer-to-peer (or distributed)network environment. The machine 900 may comprise, but not be limitedto, a server computer, a client computer, a personal computer (PC), atablet computer, a laptop computer, a netbook, a set-top box (STB), apersonal digital assistant (PDA), an entertainment media system, acellular telephone, a smart phone, a mobile device, a wearable device(e.g., a smart watch), a smart home device (e.g., a smart appliance),other smart devices, a web appliance, a network router, a networkswitch, a network bridge, or any machine capable of executing theinstructions 925, sequentially or otherwise, that specify actions to betaken by the machine 900. Further, while only a single machine 900 isillustrated, the term “machine” shall also be taken to include acollection of machines 900 that individually or jointly execute theinstructions 925 to perform any one or more of the methodologiesdiscussed herein.

The machine 900 may include processors 910, memory 930, and I/Ocomponents 950, which may be configured to communicate with each othervia a bus 905. In an example embodiment, the processors 910 (e.g., aCentral Processing Unit (CPU), a Reduced Instruction Set Computing(RISC) processor, a Complex Instruction Set Computing (CISC) processor,a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), anApplication Specific Integrated Circuit (ASIC), a Radio-FrequencyIntegrated Circuit (RFIC), another processor, or any suitablecombination thereof) may include, for example, a processor 915 and aprocessor 920 that may execute instructions 925. The term “processor” isintended to include multi-core processors that may comprise two or moreindependent processors (also referred to as “cores”) that may executeinstructions contemporaneously. Although FIG. 9 shows multipleprocessors 910, the machine 900 may include a single processor with asingle core, a single processor with multiple cores (e.g., a multi-coreprocessor), multiple processors with a single core, multiple processorswith multiples cores, or any combination thereof.

The memory 930 may include a main memory 918, a static memory 940, and astorage unit 945 accessible to the processors 910 via the bus 905. Thestorage unit 945 may include a machine-readable medium 947 on which arestored the instructions 925 embodying any one or more of themethodologies or functions described herein. The instructions 925 mayalso reside, completely or at least partially, within the main memory918, within the static memory 940, within at least one of the processors910 (e.g., within the processor's cache memory), or any suitablecombination thereof, during execution thereof by the machine 900.Accordingly, the main memory 918, the static memory 940, and theprocessors 910 may be considered machine-readable media 947.

As used herein, the term “memory” refers to a machine-readable medium947 able to store data temporarily or permanently, and may be taken toinclude, but not be limited to, random-access memory (RAM), read-onlymemory (ROM), buffer memory, flash memory, and cache memory. While themachine-readable medium 947 is shown in an example embodiment to be asingle medium, the term “machine-readable medium” should be taken toinclude a single medium or multiple media (e.g., a centralized ordistributed database, or associated caches and servers) able to storeinstructions 925. The term “machine-readable medium” shall also be takento include any medium, or combination of multiple media, that is capableof storing instructions (e.g., the instructions 925) for execution by amachine (e.g., the machine 900), such that the instructions, whenexecuted by one or more processors of the machine (e.g., the processors910), cause the machine to perform any one or more of the methodologiesdescribed herein. Accordingly, a “machine-readable medium” refers to asingle storage apparatus or device, as well as “cloud-based” storagesystems or storage networks that include multiple storage apparatus ordevices. The term “machine-readable medium” shall accordingly be takento include, but not be limited to, one or more data repositories in theform of a solid-state memory (e.g., flash memory), an optical medium, amagnetic medium, other non-volatile memory (e.g., Erasable ProgrammableRead-Only Memory (EPROM)), or any suitable combination thereof. The term“machine-readable medium” specifically excludes non-statutory signalsper se.

The I/O components 950 may include a wide variety of components toreceive input, provide and/or produce output, transmit information,exchange information, capture measurements, and so on. It will beappreciated that the I/O components 950 may include many othercomponents that are not shown in FIG. 9. In various example embodiments,the I/O components 950 may include output components 952 and/or inputcomponents 954. The output components 952 may include visual components(e.g., a display such as a plasma display panel (PDP), a light emittingdiode (LED) display, a liquid crystal display (LCD), a projector, or acathode ray tube (CRT)), acoustic components (e.g., speakers), hapticcomponents (e.g., a vibratory motor), other signal generators, and soforth. The input components 954 may include alphanumeric inputcomponents (e.g., a keyboard, a touch screen configured to receivealphanumeric input, a photo-optical keyboard, or other alphanumericinput components), point-based input components (e.g., a mouse, atouchpad, a trackball, a joystick, a motion sensor, and/or anotherpointing instrument), tactile input components (e.g., a physical button,a touch screen that provides location and force of touches or touchgestures, and/or other tactile input components), audio input components(e.g., a microphone), and the like.

In further example embodiments, the I/O components 950 may includebiometric components 956, motion components 958, environmentalcomponents 960, and/or position components 962, among a wide array ofother components. For example, the biometric components 956 may includecomponents to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification, fingerprint identification, or electroencephalogram based identification), andthe like. The motion components 958 may include acceleration sensorcomponents (e.g., accelerometer), gravitation sensor components,rotation sensor components (e.g., gyroscope), and so forth. Theenvironmental components 960 may include, for example, illuminationsensor components (e.g., photometer), temperature sensor components(e.g., one or more thermometers that detect ambient temperature),humidity sensor components, pressure sensor components (e.g.,barometer), acoustic sensor components (e.g., one or more microphonesthat detect background noise), proximity sensor components (e.g.,infrared sensors that detect nearby objects), and/or other componentsthat may provide indications, measurements, and/or signals correspondingto a surrounding physical environment. The position components 962 mayinclude location sensor components (e.g., a Global Position System (GPS)receiver component), altitude sensor components (e.g., altimeters and/orbarometers that detect air pressure, from which altitude may bederived), orientation sensor components (e.g., magnetometers), and thelike.

Communication may be embodied using a wide variety of technologies. TheI/O components 950 may include communication components 964 operable tocouple the machine 900 to a network 980 and/or to devices 970 via acoupling 982 and a coupling 992 respectively. For example, thecommunication components 964 may include a network interface componentor another suitable device to interface with the network 980. In furtherexamples, communication components 964 may include wired communicationcomponents, wireless communication components, cellular communicationcomponents, Near Field Communication (NFC) components, Bluetooth®components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and othercommunication components to provide communication via other modalities.The devices 970 may be another machine and/or any of a wide variety ofperipheral devices (e.g., a peripheral device coupled via a UniversalSerial Bus (USB)).

Moreover, the communication components 964 may detect identifiers and/orinclude components operable to detect identifiers. For example, thecommunication components 964 may include Radio Frequency Identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as Universal Product Code (UPC) bar code,multi-dimensional bar codes such as Quick Response (QR) code, Azteccode, Data Matrix, Dataglyph, MaxiCode, PDF48, Ultra Code, UCC RSS-2Dbar code, and other optical codes), acoustic detection components (e.g.,microphones to identify tagged audio signals), and so on. In addition, avariety of information may be derived via the communication components964, such as location via Internet Protocol (IP) geo-location, locationvia Wi-Fi® signal triangulation, location via detecting an NFC beaconsignal that may indicate a particular location, and so forth.

Transmission Medium

In various example embodiments, one or more portions of the network 980may be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a Wi-Fi®network, another type of network, or a combination of two or more suchnetworks. For example, the network 980 or a portion of the network 980may include a wireless or cellular network and the coupling 982 may be aCode Division Multiple Access (CDMA) connection, a Global System forMobile communications (GSM) connection, or another type of cellular orwireless coupling. In this example, the coupling 982 may embody any of avariety of types of data transfer technology, such as Single CarrierRadio Transmission Technology (1×RTT), Evolution-Data Optimized (EVDO)technology, General Packet Radio Service (GPRS) technology, EnhancedData rates for GSM Evolution (EDGE) technology, third GenerationPartnership Project (3GPP) including 3G, fourth generation wireless (4G)networks, Universal Mobile Telecommunications System (UMTS), High SpeedPacket Access (HSPA), Worldwide Interoperability for Microwave Access(WiMAX), Long Term Evolution (LTE) standard, others defined by variousstandard-setting organizations, other long range protocols, or otherdata transfer technology.

The instructions 925 may be transmitted and/or received over the network980 using a transmission medium via a network interface device (e.g., anetwork interface component included in the communication components964) and utilizing any one of a number of well-known transfer protocols(e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions925 may be transmitted and/or received using a transmission medium viathe coupling 992 (e.g., a peer-to-peer coupling) to the devices 970. Theterm “transmission medium” shall be taken to include any intangiblemedium that is capable of storing, encoding, or carrying theinstructions 925 for execution by the machine 900, and includes digitalor analog communications signals or other intangible media to facilitatecommunication of such software.

Furthermore, the machine-readable medium 947 is non-transitory (in otherwords, not having any transitory signals) in that it does not embody apropagating signal. However, labeling the machine-readable medium 947“non-transitory” should not be construed to mean that the medium 947 isincapable of movement; the medium 947 should be considered as beingtransportable from one physical location to another. Additionally, sincethe machine-readable medium 947 is tangible, the medium 947 may beconsidered to be a machine-readable device.

TERM USAGE

Throughout this specification, plural instances may be embodied ascomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be embodied as a combined structure or component. Similarly,structures and functionality presented as a single component may beembodied as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Although an overview of the inventive subject matter has been describedwith reference to specific example embodiments, various modificationsand changes may be made to these embodiments without departing from thebroader scope of embodiments of the present disclosure. Such embodimentsof the inventive subject matter may be referred to herein, individuallyor collectively, by the term “invention” merely for convenience andwithout intending to voluntarily limit the scope of this application toany single disclosure or inventive concept if more than one is, in fact,disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Moreover, plural instances may be provided forresources, operations, or structures described herein as a singleinstance. Additionally, boundaries between various resources,operations, modules, engines, and data stores are somewhat arbitrary,and particular operations are illustrated in a context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within a scope of various embodiments of thepresent disclosure. In general, structures and functionality presentedas separate resources in the example configurations may be embodied as acombined structure or resource. Similarly, structures and functionalitypresented as a single resource may be embodied as separate resources.These and other variations, modifications, additions, and improvementsfall within a scope of embodiments of the present disclosure asrepresented by the appended claims. The specification and drawings are,accordingly, to be regarded in an illustrative rather than a restrictivesense.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the possible embodiments to the precise forms disclosed. Manymodifications and variations are possible in view of the aboveteachings. The embodiments were chosen and described in order to bestexplain the principles involved and their practical applications, tothereby enable others skilled in the art to best utilize the variousembodiments with various modifications as are suited to the particularuse contemplated.

It will also be understood that, although the terms “first,” “second,”etc. may be used herein to describe various elements, these elementsshould not be limited by these terms. These terms are only used todistinguish one element from another. For example, a “first contact”could be termed a “second contact,” and, similarly, a “second contact”could be termed a “first contact,” without departing from the scope ofthe present embodiments. The first contact and the second contact areboth contacts, but they are not the same contact.

The terminology used in the description of the embodiments herein is forthe purpose of describing particular embodiments only and is notintended to be limiting. As used in the description of the embodimentsand the appended claims, the singular forms “a,” “an,” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will also be understood that the term “and/or”as used herein refers to and encompasses any and all possiblecombinations of one or more of the associated listed items. It will befurther understood that the terms “comprises” and/or “comprising,” whenused in this specification, specify the presence of stated features,integers, steps, operations, elements, and/or components, but do notpreclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof.

As used herein, the term “if” may be construed to mean “when” or “upon”or “in response to determining” or “in response to detecting,” dependingon the context. Similarly, the phrase “if it is determined” or “if (astated condition or event) is detected” may be construed to mean “upondetermining (the stated condition or event)” or “in response todetermining (the stated condition or event)” or “upon detecting (thestated condition or event)” or “in response to detecting (the statedcondition or event),” depending on the context.

This written description uses examples to disclose the inventive subjectmatter, including the best mode, and also to enable any person skilledin the art to practice the inventive subject matter, including makingand using any devices or systems and performing any incorporatedmethods. The patentable scope of the inventive subject matter is definedby the claims, and may include other examples that occur to thoseskilled in the art. Such other examples are intended to be within thescope of the claims if they have structural elements that do not differfrom the literal language of the claims, or if they include equivalentstructural elements with insubstantial differences from the literallanguage of the claims.

What is claimed is:
 1. A method comprising: storing a list of commandsin memory associated with a personal electronic device, wherein eachstored command includes a function executable by the personal electronicdevice and an associated input that causes the command to be executed;receiving user input through a touchscreen associated with the personalelectronic device; detecting one or more touch input anomalies in thereceived user input; and in response to detecting one or more touchanomalies in the received user input: changing the operating mode of thepersonal electronic device from a touch input mode to a relative motioninput mode; and altering a displayed user interface to a relative motioninput-based user interface such that the user interface in the relativemotion mode is divided into a plurality of user interface zones;receiving relative motion input, wherein the relative motion input isreceived through one or more relative motion sensors of the personalelectronic device; determining whether the received relative motioninput matches input associated with a respective command in a list ofreference commands; in accordance with a determination that the receivedrelative motion input matches input is associated with a respectivecommand in the list of reference commands, executing the respectivecommand; and in accordance with a determination that the receivedrelative motion input does not match stored input associated with arespective command in the list of reference commands, determining andexecuting a closest matching command.
 2. The method of claim 1, whereinbefore receiving relative motion input, a first user interface zone ofthe plurality of user interface zones is visually distinguished asselected, and wherein executing the respective-command comprises:changing the visually distinguished interface zone from the first userinterface zone to a second user interface zone based on the respectivecommand.
 3. The method of claim 1, wherein relative motion inputrepresents a change in the personal electronic devices position orrotation.
 4. The method of claim 1, wherein relative motion inputrepresents motion of a user relative to the personal electronic device.5. The method of claim 1, wherein the display associated with thepersonal electronic device is a touchscreen.
 6. The method of claim 1,wherein the relative motion sensors of the personal electronic deviceinclude one or more of a gyroscope, an accelerometer, a microphone, anda camera.
 7. The method of claim 1, further comprising, after executingthe closest matching command, monitoring future user inputs to determinewhether an undo command was received; and in accordance with adetermination that no undo command was received, updating input dataassociated with the closest matching command to include the receiveduser input.
 8. The method of claim 7, wherein input data associated witha stored command includes a range of possible inputs.
 9. The method ofclaim 8, wherein updating the input data associated with the closestmatching command to include the received user input further comprisesextending the range of possible inputs to cover the received user input.10. The method of claim 1, further including, prior to changing theoperating mode of the personal electronic device from the touch inputmode to the relative motion input mode, displaying an operating modechange notification on the display associated with the personalelectronic device.
 11. The method of claim 1, further including, priorto changing the operating mode of the personal electronic device fromthe touch input mode to the relative motion input mode, displaying aprompt message on the display associated with the personal electronicdevice, wherein the prompt message suggests a change of operating mode;receiving input indicating user acceptance of the change of operatingmode; and in response to receiving the input indicating user acceptanceof the change of operating mode, changing the operating mode of thepersonal electronic device from the touch input mode to the relativemotion input mode.
 12. An electronic device comprising: a storage moduleto store a list of commands in memory associated with a personalelectronic device, wherein each stored command includes a functionexecutable by the electronic device and associated input that causes thecommand to be executed; an error response module to: receive user inputthrough a touchscreen associated with the personal electronic device;detect one or more touch input anomalies in the received user input; andin response to detecting one or more touch anomalies in the receiveduser input: change the operating mode of the personal electronic devicefrom a touch input mode to a relative motion input mode; and alter adisplayed user interface to a relative motion input-based user interfacesuch that the user interface in the relative motion mode is divided intoa plurality of user interface zones; a reception module to receiverelative motion input, wherein the relative motion input is receivedthrough one or more relative motion sensors of the personal electronicdevice; an input analysis module to determine whether the receivedrelative motion input matches input associated with a respective commandin a list of reference commands; and an execution module to, inaccordance with a determination that the received relative motion inputmatches input is associated with a respective command in the list ofreference commands, execute the respective command, and in accordancewith a determination that the received relative motion in out does notmatch stored input associated with a respective command in the list ofreference commands, determining and executing a closest matchingcommand.
 13. The system of claim 12, wherein before receiving relativemotion input, a first user interface zone of the plurality of userinterface zones is visually distinguished as selected, and furthercomprising: a change module to change the visually distinguishedinterface zone from the first user interface zone to a second userinterface zone based on the respective command.
 14. The electronicdevice of claim 12, wherein the relative motion sensors of theelectronic device include one or more of a gyroscope, an accelerometer,a microphone, and a camera.
 15. A non-transitory computer-readablestorage medium storing instructions that, when executed by the one ormore processors of a machine, cause the machine to perform operationscomprising: storing a list of commands in memory associated with apersonal electronic device, wherein each stored command includes afunction executable by the personal electronic device and associatedinput that causes the command to be executed; receiving user inputthrough a touchscreen associated with the personal electronic device;detecting one or more touch input anomalies in the received user input;and in response to detecting one or more touch anomalies in the receiveduser input; changing the operating mode of the personal electronicdevice from a touch input mode to a relative motion input mode; andaltering a displayed user interface to a relative motion input-baseduser interface such that the user interface in the relative motion modeis divided into a plurality of user interface zones; receiving relativemotion input, wherein the relative motion input is received through oneor more relative motion sensors of the personal electronic device;determining whether the received relative motion input matches inputassociated with a respective command in a list of reference commands; inaccordance with a determination that the received relative motion inputmatches input is associated with a respective command in the list ofreference commands, executing the respective command; and in accordancewith a determination that the received relative motion in out does notmatch stored input associated with a respective command in the list ofreference commands, determining and executing a closest matchingcommand.
 16. The non-transitory computer-readable storage medium ofclaim 15, wherein before receiving relative motion input, a first userinterface zone of the plurality of user interface zones is visuallydistinguished as selected, and wherein executing the respective commandcomprises: changing the visually distinguished interface zone from thefirst user interface zone to a second user interface zone based on therespective command.
 17. The non-transitory computer-readable storagemedium of claim 16, wherein the relative motion sensors of the personalelectronic device include one or more of a gyroscope, an accelerometer,a microphone, and a camera.
 18. The method of claim 1, wherein in thetouch input mode, the displayed user interface displays a standard userinterface for an application displayed.
 19. The electronic device ofclaim 12, wherein in the touch input mode, the displayed user interfacedisplays a standard user interface for an application displayed.
 20. Thenon-transitory computer-readable storage medium of claim 15, wherein inthe touch input mode, the displayed user interface displays a standarduser interface for an application displayed.